This position paper argues that there is an urgent need to restructure markets for the information that goes into AI systems. Specifically, producers of information goods (such as journalists, researchers, and creative professionals) need to be able to collectively bargain with AI product builders in order to receive reasonable terms and a sustainable return on the informational value they contribute. We argue that without increased market coordination or collective bargaining on the side of these primary information producers, AI will exacerbate a large-scale "information market failure" that will lead not only to undesirable concentration of capital, but also to a potential "ecological collapse" in the informational commons. On the other hand, collective bargaining in the information economy can create market frictions and aligned incentives necessary for a pro-social, sustainable AI future. We provide concrete actions to support a coalition-based approach to achieve this goal. For example, researchers and developers can establish technical mechanisms such as federated data management tools and explainable data value estimation techniques to inform and facilitate collective bargaining in the information economy. Additionally, regulatory and policy interventions may be introduced to support trusted data intermediary organizations representing guilds or syndicates of information producers.

This position paper argues that there is an urgent need to restructure markets for the information that goes into AI systems. Specifically, small-to-medium sized producers of information (such as journalists, news organizations, researchers, and creative professionals) need to be able to appoint representatives who can carry out collective bargaining with AI product builders in order to receive a reasonable terms and a fair return on the informational value they contribute. Obstacles to this market structure can be removed through technical work that facilitates collective bargaining in the information economy (e.g., explainable data value estimation and federated data management tools) and regulatory/policy interventions (e.g., support for trusted data intermediary organizations that represent guilds or syndicates of information producers). We argue that without collective bargaining in the information economy, AI will exacerbate a large-scale information market failure that will lead not only to undesirable concentration of capital, but also to a potential ecological collapse in the informational commons. On the other hand, collective bargaining in the information economy can create market conditions necessary for a pro-social AI future. We provide concrete actions that can be taken to support a coalition-based approach to achieve this.